Apparatus for acquiring 3-dimensional maps of a scene

ABSTRACT

An active sensor for performing active measurements of a scene is presented. The active sensor includes at least one transmitter configured to emit light pulses toward at least one target object in the scene, wherein the at least one target object is recognized in an image acquired by a passive sensor; at least one receiver configured to detect light pulses reflected from the at least one target object; a controller configured to control an energy level, a direction, and a timing of each light pulse emitted by the transmitter, wherein the controller is further configured to control at least the direction for detecting each of the reflected light pulses; and a distance measurement circuit configured to measure a distance to each of the at least one target object based on the emitted light pulses and the detected light pulses.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/141,296 filed on Apr. 1, 2015, and U.S. Provisional Application No.62/160,988 filed on May 13, 2015, the contents of which are herebyincorporated by reference.

BACKGROUND

An autonomous car (also known as a driverless car, a self-driving car,or a robotic car) is a vehicle that navigates without human control. Anautonomous vehicle senses its environment to detect surroundings using,e.g., radar, Lidar, GPS, Odometer, or computer vision. Advanced controlsystems interpret sensory information to identify appropriate navigationpaths, as well as obstacles and relevant signage. Autonomous cars areequipped with control systems for analyzing sensory data in order todistinguish between different cars or obstacles on the road. Currently,driverless technology is developed by Google®, Tesla®, and some othervehicles manufactures, such as Audi®, BMW®, Nissan®, and the like.

Other companies such as, e.g., Mobileye®, are in the marketplace tryingto provide solutions for hands-free driving technology. Use of thistechnology is typically limited to particular driving infrastructuressuch as, e.g., highways or country roads, and usually only occurs whentraffic is not congested. The corner-stone of such hands-free drivingand autonomous vehicles technologies is the rendering or generation of a3-dimensional (3D) map of a scene at any given moment during orimmediately prior to motion. Such a map tries to mimic a scene as wouldhave been seen by a driver.

The rendering of such 3D-maps is typically accomplished by measuringdistances from many points in the 3D space to determine the existence ofobjects and their respective distances from the vehicle. The rendered3D-maps may be combined and processed to produce driving decisions bythe vehicle. Existing solutions for rendering detailed 3D-maps are basedon LiDar (or LADAR) systems. A LiDar system measures distance to anobject by illuminating a target (i.e., a point in space) with multiplelaser beams. Such existing solutions configure the LiDar system to scanthe entire scene. Rendering a single 3D-map therefore requires a largenumber of laser measurements.

For example, FIG. 1 shows an image 100 of a scene for which a 3D-map isgenerated. Some existing solutions implemented by hands-free andautonomous driving technologies measure the distance to each point 110in the image 100. Thus, a laser beam illuminates each such point 110 torender the 3D-map. In many instances, the LiDar system does not have anyprior knowledge of the scene, e.g., a picture of the scene. To this aim,such technologies are based on very complex and expensive equipment. Forexample, a robotic car made by Google® includes equipment worth about$150,000 including a $70,000 LiDar system. The LiDar system includes a64-beam laser. Due in part to the high cost of the hardware forrendering 3D-maps, mass production of autonomous vehicles is notfeasible. It should be noted that only a few points 110 are specificallylabeled in FIG. 1 merely for simplicity purposes.

In addition, widespread production of the autonomous vehicle usingexisting Lidar systems would create hazardous conditions to pedestrians,drivers, and/or passengers because of the high number of laser beamsthat would be transmitted from each vehicle and would likely hit aperson in the line of sight. Further, the existing LiDar solutions aretypically configured to transmit all laser beams at the highestavailable energy. This is performed to measure a point at a maximumrange of the LiDar system.

Other disadvantages of existing scanning systems (e.g., LiDar-basedsystems) is that they are usually based on moving parts, such as motors,to allow fast scanning by directing the multiple laser beams atdifferent direction. Such moving parts are expensive and prone tofailure. In addition, the resolution of scanning systems is unified asthe entire scene is scanned. There is no granularity for scanning onearea at a different (e.g., higher) resolution then another area.

Moreover, the generation of 3D-maps by scanning the entire scene tendsto increase the crosstalk. This is due to the high number of laser beamsbeing transmitted by autonomous vehicles equipped with such scanningsystems. As a result, the resulting resolution of 3D-maps generated byscanning the entire scene may be limited.

It would therefore be advantageous to provide a solution for generating3D-maps that would overcome the deficiencies of the prior art.

SUMMARY

A summary of several example embodiments of the disclosure follows. Thissummary is provided for the convenience of the reader to provide a basicunderstanding of such embodiments and does not wholly define the breadthof the disclosure. This summary is not an extensive overview of allcontemplated embodiments, and is intended to neither identify key orcritical elements of all embodiments nor to delineate the scope of anyor all aspects. Its sole purpose is to present some concepts of one ormore embodiments in a simplified form as a prelude to the more detaileddescription that is presented later. For convenience, the term “someembodiments” may be used herein to refer to a single embodiment ormultiple embodiments of the disclosure.

Certain embodiments disclosed herein include an active sensor forperforming active measurements of a scene. The scene may be a field ofview captured by an apparatus for causing generation of the 3D-map. Theactive sensor comprises at least one transmitter configured to emitlight pulses toward at least one target object in the scene, wherein theat least one target object is recognized in an image acquired by apassive sensor; at least one receiver configured to detect light pulsesreflected from the at least one target object; a controller configuredto control an energy level, a direction, and a timing of each lightpulse emitted by the transmitter, wherein the controller is furtherconfigured to control at least the direction for detecting each of thereflected light pulses; and a distance measurement circuit configured tomeasure a distance to each of the at least one target object based onthe emitted light pulses and the detected light pulses.

Certain embodiments disclosed herein also include a method forperforming active measurements of a scene. The method comprisesreceiving an identification of at least one target object recognized inan image acquired by a passive sensor; receiving a current location ofthe at least one target object; determining an energy level, adirection, and a timing of a light pulse to be transmitted to each ofthe at least one target object; setting a transmitter with thedetermined energy level, direction, and timing for emitting each lightpulse; setting a receiver to receive each light pulse reflected from theat least one target object at the determined at least one direction; andmeasuring a distance to each of the at least one target object based onthe corresponding emitted light pulse and the corresponding detectedlight pulse.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter disclosed herein is particularly pointed out anddistinctly claimed in the claims at the conclusion of the specification.The foregoing and other objects, features, and advantages of thedisclosed embodiments will be apparent from the following detaileddescription taken in conjunction with the accompanying drawings.

FIG. 1 is a picture of a scene for which a 3D-map is generated using aconventional approach.

FIG. 2 is a block diagram of an apparatus configured to generate 3D-mapsfrom a vehicle according to one embodiment.

FIGS. 3A and 3B showing identification of objects in an image accordingto an embodiment.

FIG. 4 is a block diagram of the active sensor arranged according to oneembodiment.

FIG. 5 is a schematic diagram of a static LDC constructed using a MEMSmirror according to an embodiment.

FIG. 6 is a schematic diagram of a static LDC constructed using anoptical modulator according to an embodiment.

FIG. 7 illustrates the operation of a distributed optical sensoraccording to an embodiment.

FIG. 8 is a flowchart illustrating a method for generating a 3D-map of ascene according to an embodiment.

DETAILED DESCRIPTION

It is important to note that the embodiments disclosed herein are onlyexamples of the many advantageous uses of the innovative teachingsherein. In general, statements made in the specification of the presentapplication do not necessarily limit any of the various claimedembodiments. Moreover, some statements may apply to some inventivefeatures but not to others. In general, unless otherwise indicated,singular elements may be in plural and vice versa with no loss ofgenerality. In the drawings, like numerals refer to like parts throughthe several views.

According to the disclosed embodiments, 3D-maps are rendered orotherwise generated based on active measurements, passive measurements,and/or analysis of information related to such measurements. Activemeasurements may be laser measurements, while passive measurements maybe achieved by image processing. Therefore, the disclosed embodimentsallow for provision of a complete 3D-map of a scene while using a smallnumber of low-energy laser beams. The scene includes various objects(e.g., people, inanimate objects, vehicles, animals, and the like) thatcan be represented by a 3D map. The scene may be limited in size orotherwise limited in scope. As an example, a scene may include allobjects within 20 feet of a particular point in space. In certainconfigurations, the disclosed embodiments can be utilized to generate ahigh-resolution 3D-map of certain sections of a scene without performingactive measurements for the entire scene. The utilization of a fewactive measurements reduces the risk of human exposure to laser beams.

FIG. 2 shows an example block diagram of an apparatus 200 configured togenerate 3D-maps from a vehicle according to one embodiment. Theapparatus 200 includes a memory 215, an active sensor 220, and a passivesensor 230 connected to the processing system 210. In certainconfigurations, the apparatus 200 further includes a storage 240, one ormore motion sensors 250, and a network interface (I/F) 260 connected tothe processing system 210.

The apparatus 200 can be, e.g., mounted on or integrated in a vehicle.Such a vehicle may include, for example, a car, a truck, a bus, a drone,a robot, and the like. The apparatus 200 can be utilized to generate3D-maps of the scene as would be observed by an operator (e.g., adriver) of the vehicle. In an embodiment, the apparatus 200 isconfigured to control part or all of the operation of the vehicle basedon analysis of the generated 3D-maps. Therefore, the apparatus 200 canbe utilized in applications related to autonomous vehicles, hands-freedriving systems, driver assistance systems, and the like.

The passive sensor 230 may be an image sensor (e.g., a camera)configured to passively acquire images. The acquired images can beprocessed by the processing system 210 or by the passive sensor 230using image processing techniques.

The active sensor 220 is configured to perform active measurements inspecific directions to determine the distance from objects in the scene.In an embodiment, the active measurements are laser measurements. In afurther embodiment, the active sensor 220 is configured to illuminatelight beams in specific directions (determined by, e.g., the processingsystem 210) and to detect the light beams reflected off of objects inthe scene. The distance from an object can be computed based on theenergy and reflection angle of each reflected light beam. The activesensor 220 can be realized as, for example, an active stereovision, astructured light, a time-of-flight (TOF), and the like. A detaileddiscussion of the active sensor 220 is provided below.

In an embodiment, the active sensor 220 is synchronized with the passivesensor 230 in such a way that, for each new frame (of an image acquiredby the passive sensor 230), the active sensor 220 is triggered to startscanning the frame. In another embodiment, the active sensor 220 and thepassive sensor 230 have the same resolution.

In order to provide an accurate operation for the active sensor 220 andthe passive sensor 230, a calibration process may be performed. Thecalibration process allows for accurate mapping of pixels in imagesacquired by the passive sensor 230 to pixels scanned by the activesensor 220, and vice versa.

In an embodiment, the calibration process includes causing the activesensor 220 to scan, via light beams, a portion of a scene in differentdirections to build a 3D image. The scanning laser beams are detected bythe active sensor 220 and by the passive sensor 230, and can be utilizedto create a 3D map for each of the sensors 220 and 230. The generatedmaps are compared to each other to calibrate one sensor respective ofthe other.

The processing system 210 may be configured to process data (e.g.,energy level, direction, reflection angle of the emitted and reflectedbeams, and so on) in order to generate a 3D-map of the entire scene orof certain objects in the scene. As will be discussed below, the numberof measurements captured by the active sensor may be limited to certainpoints and/or surfaces in space as the initial scanning is performedusing any images acquired by the passive sensors 230. In an embodiment,to create the 3D-map, additional information such as, but not limitedto, previously acquired images or previously generated 3D-maps,previously determined active measurements, and/or sensory informationgathered from the motion sensors 250 may be utilized. The motion sensors250 may include, but are not limited to, a global positioning system(GPS), an accelerometer, a proximity sensor, an odometer, and the like.The additional information together with the active and passivemeasurements can be utilized by the processing system 210 to determine asize of each object, its speed, and/or its direction relative to thevehicle.

According to certain embodiments, the processing system 210 isconfigured to generate 3D maps. To this end, the processing system 210may be configured to fuse active and passive measurements; to determinestationary or nearly stationary objects; to actively measure distancesonly from moving objects; and/or to estimate distances from objectsusing the top possible velocity of objects and/or using passivemeasurements.

The fusion of active and passive measurements may include, but is notlimited to, utilizing the active sensor 220 to measure objects in areaswere the passive sensor 230 does not provide a reliable measurement. Inan embodiment, if a reliable passive measurement can be achieved, theactive sensor 220 is not utilized. Thus, the disclosed embodiments maysignificantly reduce the number of active measurements performed by theactive sensor 220.

As such, fewer laser beams or other sources may be utilized, therebyreducing energy consumption, crosstalk, and hazardous conditions, aswell as achieving higher resolution. In an embodiment, the apparatus 200can be configured such that only suspicious objects will be activelymeasured using the active sensor 220. A suspicious object may be anyobject that requires active measurement to accurately generate a mapbased thereon. An object may be suspicious if, e.g., the object belongsto a predetermined list of suspicious objects. Determining whether anobject is suspicious may further include image-based analysis of theobject to identify a type of the object (e.g., tree, human, animal,vehicles, and the like). As an example, if the apparatus 200 is mountedon a car, pedestrians in the foreground of captured images will beconsidered as suspicious objects while trees in the background ofcaptured images are not. The classification of objects as suspicious isdiscussed further herein below.

In an embodiment, the processing system 210 is configured to determinewhich objects are stationary or near stationary based on images providedby the passive sensor 230. For example, using image recognition, theprocessing system 210 may be configured to determine stationary objectssuch as, e.g., a house, a tree, a pole, etc. In a further embodiment,when the passive sensor 230 is fixed (i.e., not moving), the processingsystem 210 may be configured to determine if an object is stationary bycomparing two consecutive images and determining which objects have beenmoved relative to stationary objects based on the comparison. As anexample, if the passive sensor is fixed to the side of a building in ascene, consecutive images may be compared to determine whether anyobjects in the scene have been moved relative to stationary objects. Inan embodiment, when the passive sensor 230 is in motion, the distancefrom stationary objects can be measured using, e.g., GPS coordinates. Asan example, if the passive sensor is attached to a car, the distancefrom stationary objects may be measured using GPS measurements and thespeed of the car. This may further require processing the consecutiveimages acquired by the passive sensor 230.

In yet a further embodiment, determining whether objects are stationarymay be based on comparisons of frames (e.g., images) captured by thepassive sensor 230. The comparisons may include, but are not limited to,determining changes in distances between sets of points in the frames.In an example embodiment, each distance change between two points may beequal to a difference between the 3D distance between the points as seenin a first frame with the 3D distance between the points as seen in asecond frame. Any object that is determined to have moved may beassociated with a non-stationary object. In an embodiment, an object maybe determined to have moved if, for example, a sum of all distancechanges related to the object is above a predefined threshold. Inanother embodiment, an object may be determined to have moved if the sumof distance changes related to the point is greater than the sum ofdistance changes related to each of the other objects above a predefinedthreshold. In yet another embodiment, an object may be determined tohave moved if a distance change related to an object in a subsequentframe is greater than a distance change related to the object in a priorframe above a predefined threshold.

In another embodiment, determining whether objects in images arestationary may be based on predicted locations of objects in an image. Aprediction image may be generated based on a frame of a currentlyacquired image or generated 3D-map. The prediction image may be furtherbased on a movement of the sensor acquiring the current image orproviding information used to generate the current 3D-map. Based on thecurrent frame and/or any new position or orientation of the sensor, aprediction frame illustrating predicted positions of the objects at asubsequent time (assuming the objects do not move) is generated. Theprediction frame may be compared to a subsequent frame based on sensorreadings at the subsequent time to determine if there are anydifferences between locations of the objects. In an embodiment, if adifference in location of an object between the prediction frame and acorresponding subsequent frame is above a predefined threshold, it maybe determined that the object has moved.

In an embodiment, when the passive sensor 230 is in motion, the distancefrom stationary objects can be measured using GPS and the speed of thecar. In another embodiment, the distance from stationary objects can bemeasured without using GPS by comparing frames (e.g., acquired images)of passive sensor information and/or 3D mapping information. In afurther embodiment, finding the distance and/or angle change between anytwo frames may be determined, for each pair of corresponding points inthe frames, based on a weighted score. To this end, in an embodiment,finding the distance or angle change between two frames may furtherinclude determining matching or otherwise corresponding points in theframes. In yet a further embodiment, the weighted score may be based on,but not limited to, a degree of error associated with the distance ofthe point in each frame such as, e.g., error based on noise.

In a further embodiment, a prediction frame may be generated based on 2Dor 3D mapping information as well as a location and orientation of asensor. The prediction frame may be a 3D image illustrating predictedpositions of stationary (i.e., non-moving) objects included in the3D-mapping information. Such stationary objects may include, but are notlimited to, trees, billboards, signs, buildings, and other permanent orsemi-permanent items or fixtures. The prediction frame may be utilizedto, e.g., fill in exposed gaps of a previous frame (for example, when anobject moves, the prediction frame may be utilized to illustrate aprediction for items behind the previous location of the moved object).

In another embodiment, the passive measurements may be performed basedon movements of objects classified as non-stationary and non-suspicious.To this end, passively measuring the distance may include, but is notlimited to, determining a boundary speed of the object and a directionof movement of the object. The boundary speed is a speed at which theboundary (i.e., outer surface) of the object is traveling. Based on thedetermined boundary speed and direction, a distance to the object may beestimated. In an embodiment, the estimated distance may be further basedon a previously known distance to the object and/or previously generated3D-maps including the object.

In an embodiment, the processing system 210 is configured to estimatethe distance of the apparatus 200 and/or the vehicle from non-stationaryobjects using a maximum and/or minimum possible speed of such objects.For example, if a distance from a pedestrian is initially measured, thena range (e.g., between 0 and 2 meters) representing potential distancesof the pedestrian from the apparatus 200 at a given time after theinitial measurement can be determined based on a predetermined maximumvelocity of a person (e.g., 2 meters per second).

In an embodiment, the processing system 210 will not trigger anotheractive measurement with respect to an object if the object is locatedwithin a range that does not require any action (e.g., when a pedestrianis walking in a sidewalk 100 feet across the street from the apparatus200). Whether an action is required may be based on a safe distancethreshold. The safe distance threshold may include, but is not limitedto, the speed of the vehicle and/or object, the distance of the objectfrom the apparatus 200, the direction of the object relative to theapparatus 220, a type of the object, combinations thereof, and the like.In a further embodiment, active measurements may be performed when anestimated distance for a moving object (e.g., based on a boundary speedand a direction of the object) does not meet a safe distance threshold.An action may include, for example, controlling a hands free orautonomous vehicle.

In an embodiment, the processing system 210 is configured to estimatethe distance of the apparatus 200 from non-stationary and stationaryobjects using passive measurements. Specifically, the distance may beestimated based in part on the object's approximate size and the amountof pixels the object occupies in an image acquired by the passive sensor230. Such an estimation may further include determining if the object isin close-proximity to the apparatus 200 or not. In an embodiment, forsome objects that are not in close-proximity, no active measurement bythe active sensor 220 is required. The close-proximity can be determinedbased on a predefined threshold (e.g., a distance of under 100 metersmay be determined to be in close-proximity).

Consideration of particular (e.g., suspicious) objects allows forreducing the number of active measurements by the active sensor 220.

Further consideration of objects is demonstrated in reference to FIGS.3A and 3B, each showing an example image of a scene for which a 3D-mapis generated according to the disclosed embodiments. As an example, animage 300 of the scene is acquired by the passive sensor 230. The image300 may be segmented to identify objects 310. The segmentation andidentification of objects can be performed using conventional imageprocessing techniques. For example, the road, pedestrian, car, van,track, crash barrier, and noise barrier may be identified as the objects310. Then, object classification and passive measurements are performed.Based on the object classification, it is determined which activemeasurements should be made, that is, where to direct the laser beamsemitted by the active sensor 220. The classification of objects mayinclude, but is not limited to, determining whether objects in a sceneare stationary, non-stationary, and/or suspicious. For example, thepedestrian, the car, the van, and the track may be classified asnon-stationary and suspicious objects and, as such, active measurementsin many different directions are triggered at locations 320 (shown inFIG. 3B) based, in part, on the object classification. The road, crashbarrier, sky, road, and sound barrier wall may be classified asstationary and non-suspicious objects and, accordingly, fewer activemeasurements are performed.

As demonstrated by FIG. 3B, only a portion of the scene acquired in theimage 300 is scanned with the active sensor 220. The locations foractive measurements are labeled as 320. Performing a greater number ofactive measurements on a limited number of objects allows for providinga higher resolution with respect to these objects. This further enablesproviding higher resolution in certain areas where detail recognition isrequired.

It should be noted that FIGS. 3A and 3B are merely examples and do notlimit the various disclosed embodiments. In particular, more, fewer, ordifferent objects 310 and/or locations 320 may be identified for imageswithout departing from the scope of the disclosure.

Once all measurements are completed, a 3D-map of the scene may begenerated. Specifically, the 3D-map is generated by computing thedistance of the apparatus from any pixel in the acquired image. That is,in an embodiment, generating the 3D-map may include associating eachpixel with a distance value. The distance values may be derived from thepassive measurements and/or active measurements. In an embodiment, onlya limited set of distance measurements are performed (e.g., fewermeasurements than the number of pixels). The distance values can becomputed or extrapolated using a plane equation or other equivalenttechniques.

Returning to FIG. 2, the processing system 210 may comprise or be acomponent of a processor (not shown) or an array of processors coupledto the memory 215. The memory 215 contains instructions that can beexecuted by the processing system 210. The instructions, when executedby the processing system 210, cause the processing system 215 to performthe various functions described herein. The one or more processors maybe implemented with any combination of general-purpose microprocessors,multi-core processors, microcontrollers, digital signal processors(DSPs), field programmable gate array (FPGAs), programmable logicdevices (PLDs), controllers, state machines, gated logic, discretehardware components, dedicated hardware finite state machines, or anyother suitable entities that can perform calculations or othermanipulations of information.

The processing system 210 may also include machine-readable media forstoring software. Software shall be construed broadly to mean any typeof instructions, whether referred to as software, firmware, middleware,microcode, hardware description language, or otherwise. Instructions mayinclude code (e.g., in source code format, binary code format,executable code format, or any other suitable format of code). Theinstructions, when executed by the one or more processors, cause theprocessing system to perform the various functions described herein.

The storage 240 may be magnetic storage, optical storage, and the like,and may be realized, for example, as flash memory or other memorytechnology, CD-ROM, Digital Versatile Disks (DVDs) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tostore the desired information.

In one configuration, computer readable instructions for implementingany of the embodiments disclosed herein may be stored in the storage240. The storage 240 may also store other computer readable instructionsto implement an operating system, an application program, and the like.Computer readable instructions may be loaded in the memory 215 forexecution by the processing system 210. The storage 240 may also beconfigured to store, for example, images, generated 3D-maps, 3D or 2Dmaps provided by mapping services (e.g., street-level maps), and thelike.

The network interface 260 may allow the apparatus 200 to communicatewith other apparatuses such as, but not limited to, a controller of thevehicle (not shown), a central controller (not shown), and a cloudstorage device (not shown). For example, the network interface 260 maybe configured to allow the apparatus 200 to communicate with thecontroller of the vehicle to provide operating instructions (e.g., stop,turn right, etc.) over a network (not shown). In an embodiment, thenetwork interface 260 allows remote access to the apparatus 200 for thepurpose of, for example, configuration, loading of new maps, and so on.The network interface 260 may include a wired connection or a wirelessconnection. The network interface 260 may transmit and/or receivecommunication media. For example, the network interface 260 may includea modem, a network interface card (NIC), an integrated networkinterface, a radio frequency transmitter/receiver, an infrared port, aUSB connection, and the like.

FIG. 4 is an example block diagram of the active sensor 220 arrangedaccording to one embodiment. The active sensor 220 includes atransmitter (TX) 410 and a receiver 420 connected to a controller 430.In an optional configuration, the active sensor 220 also includes adistance measurement circuit (DMC) 440. In one embodiment, thecontroller 430 may be realized as the processing system 210. In anotherembodiment, the controller 430 is a separate element from the processingsystem 210 and can be realized as, but not limited to, general-purposemicroprocessors, multi-core processors, microcontrollers, DSPs, FPGAs,PLDs, controllers, state machines, gated logic, discrete hardwarecomponents, dedicated hardware finite state machines, or any othersuitable entities that can perform calculations or other manipulationsof information. It should be noted that, in certain configurations, theactive sensor 220 may include a plurality of transmitters 410 and/or aplurality of receivers 420.

In an embodiment, the transmitter 410 is configured to emit or transmitlight pulses (e.g., laser beams) at varying energy levels and atdifferent directions. The energy level, direction, and timing of eachlight pulse may be determined by the controller 430. Multiple lightpulses can be sent at different directions at roughly the same time. Theenergy levels of such light pulses may be the same or different.

As noted above, the active sensor 220 and/or a target object may be inmotion when the active measurements are performed. A measurement to aspecific point (a pixel in an identified object) is determinedrespective of the analysis of the image captured by the passive sensor.The location of such point is likely to be moved from when the image iscaptured and processed until the active measurement is triggered (i.e.,a light pulse is emitted). To compensate for such a difference, the newlocation of the point is estimated. To this end, the timing, directionand/or energy level of the light pulse may be determined, with theestimated new location based on the determined timing, direction, and/orenergy level.

The new location of the point to direct the active measurement to can beestimated based on the time elapsed since the image is captured untilthe measurement is triggered, information provided by one or more motionsensors (e.g., accelerometer, GPS, and/or gyroscope), previouslycaptured images, and/or previously generated 3D maps. For example,gyroscope information can be used to determine an orientation of thesensor 220 at a new location respective of the old location (i.e., thelocation at which the image was captured).

In an embodiment, the transmitter 410 includes a light source 412, astatic light directing circuit (LDC) 414, and one or more transmit (TX)lenses 416 utilized to focus the energy of the light pulses. The lightsource 412 may include any device that radiates or emits electromagneticradiation in multiple wavelengths or in a single wavelength. The lightsource 412 may be realized as, but not limited to, a laser, a LED, anarray of LEDs, and the like.

In an embodiment, the light source 412 is configured to emit a lightpulse at an energy level determined based on the maximum distance to anobject that the pulse is directed to. The energy level may be controlledby the controller 430.

The static LDC 414 is configured to direct the light pulse emitted bythe light source 412 in a particular direction. The static LDC 414 canbe implemented without any moving parts. In one embodiment, the staticLDC 414 is realized using a microelectromechanical system (MEMS) mirroror an array of MEMS mirrors. When the static LDC 414 is realized using asingle MEMS mirror, the scanning may be performed along both the X and Yaxes by the same mirror. Alternatively, the static LDC 414 is realizedusing two MEMS mirrors, each MEMS mirror be can used to scan along adifferent axis. In another embodiment, the static LDC 414 is realizedusing an acousto-optic modulator (AOM). In yet another embodiment, thestatic LDC 414 is realized using an electro-optic modulator (EOM).Different realizations of the static LDC 414 may require differentconfigurations.

A MEMS mirror is an optical beam steering device. When using a MEMSmirror in the static LDC 414, a light pulse (or beam) can be steered toa specific direction and/or split into different directions. In thisconfiguration, the MEMS mirror is horizontally placed in the LDC 414.This is further demonstrated in FIG. 5, showing the static LDC 414constructed using a MEMS mirror 510.

In this configuration, a light beam 501 hits the MEMS mirror 510 at onedirection and is reflected at a different direction controlled by thecontroller 430 (controller 430 not shown in FIG. 5). Specifically, theMEMS mirror 510 can be tilted along the X and Y axes, where the tiltingmay be caused by applying an electric signal (generated by, e.g., thecontroller 430) along the X and Y axes. This signal causes changing ofthe tilt of the mirror 510 at a predefined range of spatial angles. Asshown in FIG. 5, one light beam 501 can be reflected from the MEMSmirror 510 in different directions (illustrated as in FIG. 5 as examplereflected beams 502, 503, and 504, respectively).

In another embodiment, a constant electric signal is applied, therebycausing the MEMS mirror 510 to tilt at a predefined range of spatialangles. In this embodiment, the transmission of the light beam is timedaccording to the angle that the mirror is at. This allows fortransmission of the beam at a direction without causing any sudden orrapid changes and/or waiting for the MEMS mirror 510 to stabilize at therequested direction. In certain configurations, the MEMS mirror 510 canbe replaced with other types of scanning mirrors.

An EOM is an optical device in which a signal-controlled elementexhibiting the electro-optic effect is used to modulate and steer alight beam. An AOM is an optical device in which the acousto-opticeffect is utilized to modulate and steer a light beam. An electricsignal drives the transducer to vibrate, which creates sound waves inthe material, thereby causing modulation in the light beam.

When using an AOM or EOM in the static LDC 414, a light beam (or pulse)can be steered to a specific direction and/or split into differentdirections. In this configuration, the AOM or EOM may be placedvertically in the LDC 414. This is further demonstrated in FIG. 6,showing the static LDC 414 constructed using an optical modulator 610.The optical modulator 610 may be an AOM or an EOM.

In this configuration, the light beam 601 hits the mirror 610 at onedirection and is reflected in different directions as controlled by,e.g., the controller 430 (controller 430 not shown in FIG. 6).Specifically, the controller 430 generates an electrical signal input tothe optical modulator 610, and such a signal causes the modulator 610 tosteer the beam 601 in different directions. As shown in FIG. 6, onelight beam 601 can be reflected from the MEMS mirror 610 in differentdirections (illustrated as example reflected beams 602, 603, and 604,respectively).

Using the above-mentioned configuration, the static LDC 414 can directthe light beams at multiple spatial angles. In an embodiment, the staticLDC 414 is synchronized with the light source 412 so as to direct eachemitted light at a different spatial angle. In another embodiment, thestatic LDC 414 is controlled to rapidly change between angles to performa large number of active measurements (using the active sensor) in eachframe. For example, in each frame of 30 msec, about 30,000 activemeasurements may be performed. The angles or directions to direct thelight beam may be preordered in a specific order to achieve maximum areascanning during a predefined time interval.

In an embodiment, the lenses 416 are collimating lenses. For single andmulti-mode lasers with cylindrical symmetry, the collimating lenses 416may magnify the beam waist of the laser and may transform the laser beamangular distribution into the far field. In another embodiment, thecylindrical lenses may equalize the spread angles in both directions.

In an embodiment, the receiver 420 is configured to detect light signalsreflected from a target object to allow active measurements of adistance to an object. In an embodiment, the transmitter 410 transmitslight pulses that hit a target and a portion of each pulse's energy isreflected back toward the receiver 420. The reflected signals aredetected and the time elapsed between emission of the respective lightpulse and the detection of the returned signal is determined. In afurther embodiment, the DMC 440 measures the distance to the objectbased on the elapsed time.

In an optional embodiment, the DMC 440 measures the distance based onthe phase delay between the transmitted and reflected (received) lightsignals. In this embodiment, the transmitted light pulses aresinusoidally modulated at varying frequencies or at a single frequency.

In one embodiment, the receiver 420 includes a detector 422 and lenses424. Optionally, the receiver 420 also includes a static LDC 426. Thedetector 422, in an embodiment, is realized as an array of photodiodes.In another configuration, the detector 422 is realized as a chargecoupled device (CCD) or as one or more complementary metal oxidesemiconductor (CMOS) light sensors. The configuration of the receiver420 depends on the type of the light source.

In an embodiment, to expedite the acquisition of the activemeasurements, the transmitter 410 is configured to transmit multiplelight pulses at different directions simultaneously, thereby allowingfor measurement of different distances based on the simultaneous lightpulses. To this end, the receiver 420 is configured to detect allreflected signals in parallel.

In another embodiment, the detector 422 receives light signals reflectedfrom specific directions after hitting the target object. This allowsfor reduction of the crosstalk and the optical noises by filtering outsignals from other directions. To this end, the static LDC 426 iscontrolled, by the controller 430, to direct signals reflected fromspecified directions to the detector 422. As noted above, a static LDC426 can be realized as a MEMS mirror, an array of MEMS mirrors, an AOM,an EOM, and the like.

The lenses 424 may also be designed to reduce the crosstalk and toincrease the signal to noise ratio (SNR). To this end, the lenses 424are typically either infrared filters or polarized lenses. If the lenses424 are polarized lenses, in an embodiment, the light source 412 maysend polarized light. Reducing the crosstalk is important when, forexample, many vehicles equipped with scanning capabilities are drivingon the same road and, in particular, when those vehicles are close toeach other. Increasing the SNR allows transmission of low energy lightpulses and efficient detection of reflections thereof.

In an embodiment, the lenses 424 include a polarized lens which reducesoptical noise and crosstalk. In a further embodiment, the lenses 416 mayalso include a polarizer with the same angle as the lenses 424. In anembodiment, the predefined angle is 45 degrees. In an embodiment, eachactive sensor 220 only detects light signals that were generated by itsown transmitter 220, while signals generated by other apparatuses(installed on different vehicles) may be filtered. Therefore, theconfiguration of lenses 424 and 416 with polarized lenses wouldeliminate any light signals transmitted by other vehicles driving in,for example, the opposite direction.

The active sensor 220 discussed with reference to FIG. 4 can beimplemented as a single component. The single component may be realizedas a chipset, an ASIC, and the like. In another embodiment, the activesensor 220 may be implemented as a distributed component including oneor more transmitters 410 and receivers 420, controlled by multiplecontrollers or a single centralized controller 430. Further, in such anembodiment, the same optics can be shared between the transmitter andreceiver. That is, the optical signal can be received and transmittedthrough the same optical path.

In one distributed configuration of the active sensor 220, thetransmitter or transmitters are stationary (e.g., installed on poles onthe side of a road) while the receiver or receivers are non-stationary(e.g., installed on the moving vehicles). This distributed configurationis further illustrated in FIG. 7, which shows an example bird's eye viewmap utilized to describe an embodiment.

The transmitters 710-1 through 710-3 (hereinafter referred toindividually as a transmitter 710 and collectively as transmitters 710,merely for simplicity purposes) are placed along the road at locationscovering one lane and do not interfere with signals transmitted at theopposite lane. To measure a distance, for example, between cars, thetimes at which the light pulse was transmitted and the times at whichthe light pulse was received, as well as the locations of thetransmitters 710 and one or more of the cars should be known. Thereceivers (not shown in FIG. 7) may be installed on any or all of thevehicles 720-1 through 720-6.

As an example, in order to measure the distance between the vehicles720-1 and 720-2, the locations of the transmitter 710-1 and the vehicle720-1, the time at which the light pulse 701 was transmitted by thetransmitter 710-1, and the time at which the retuned signal 702 wasdetected by the receiver (not shown) in the vehicle 720-1 aredetermined. The location of the vehicle 720-1 can be provided by a GPSin the vehicle 720-1.

The light pulse 701 travels from the transmitter 710-1 to the vehicle720-2, and is returned as a signal 702 from the vehicle 720-2 to thevehicle 720-1. Based on the location of the vehicle 720-1 and thelocation of the transmitter 710-1 as well as the time at which the lightpulse 701 was transmitted, the distance between the vehicle 720-1 andthe vehicle 720-2 can be determined, for example, by the receiver in thevehicle 720-1. The information related to time and locations can beshared using communication protocols.

In another distributed configuration of the active sensor 220, thetransmitters and the receivers are non-stationary but are separated.Thus, the transmitters and receivers may be installed on, for example,different vehicles. Each transmitter is configured to cover a section ofa scene (e.g., a road) and, therefore, each object in the coveredsection is illuminated only once by the transmitter at a point that isclose to the object. Thus, the transmitted light pulses may be at alower energy level. The measurement of the distances is performed asdescribed further herein above with respect to the distributedconfiguration. The location of the transmitters, in this case, can beprovided by a GPS.

FIG. 8 is an example flowchart 800 illustrating a method for generating3D-maps of a scene according to an embodiment. In an embodiment, themethod may be performed by an apparatus (e.g., the apparatus 200)operable in, for example, a vehicle, and the generated 3D-map mayrepresent the scene as it would have been seen by an operator of thevehicle. The 3D-map may be generated based on a fusion of passive andactive measurements.

At S810, one or more images is acquired by a passive sensor. In anembodiment, the acquired images are saved in a memory according to theiracquisition order. In another embodiment, the acquired images mayinclude frames from a video being continually recorded.

At optional S820, each image may be segmented to generate a segmentationmap. The segmentation map includes a plurality of segments, each ofwhich features homogeneous pixels having the same color or substantiallysimilar colors.

At S830, objects are identified in the images and/or segmentation map.The objects may be identified based on, e.g., image recognition. Itshould be noted that an object can include multiple segments. Forexample, a track colored with two different colors can be segmented astwo segments, but identified as one object.

At S840, the identified objects are classified. In an embodiment, theobjects may be classified into one or more of the following categories:stationary (i.e., immobile), non-stationary (i.e. moving or likely tomove), suspicious, or non-suspicious. In an embodiment, whether anobject is non-stationary or stationary may be determined by identifyingthe object (e.g., using image recognition) and based in part or entirelyon its type. For example, an object showing a building may be classifiedas stationary. In another embodiment, the non-stationary or stationarycategory of an object is determined by comparing two consecutive imagestaken during a predefined time interval to determine if the location ofan object has changed. If the location of the object remains the same,the object is classified as stationary; otherwise, the object may beclassified as non-stationary. In another embodiment, the objectclassification may be based on the size of the identified object. As anon-limiting example, an object having a size above a predefinedthreshold (e.g., an object large enough that a vehicle would need tonavigate around the object) may be classified as suspicious, but anobject having a size below the predefined threshold (e.g., an objectsmall enough that the vehicle can ride over the object safely) may beclassified as non-suspicious.

In an embodiment, the classification based on the image comparisons maybe further based on movement (or lack thereof) and/or positioning of thepassive sensor. Specifically, when the passive sensor is not in motion(for example, if the passive sensor is attached to a vehicle that isstopped at a traffic light) and the two consecutive images are takenfrom the same angle, the comparisons may indicate changes in thelocation of a moving object relative to other objects. When the passivesensor itself is in motion (for example, if the passive sensor isattached to a moving vehicle), the two consecutive images are taken fromdifferent angles. To allow an accurate comparison between the differentangle images, in an embodiment, a second image (e.g., the subsequentlytaken image) may be transformed to match an angle of a first image(e.g., a previously taken image), thereby placing the two images at thesame angle. The image transformation can be performed using a virtualcamera technique. It should be noted that, in one scene, an object canbe classified as stationary (e.g., a parked car), while in the nextscene, the object may be classified as non-stationary (e.g., a carmoving from a parking spot). In an embodiment, the classification ofstationary objects can be performed based in part on 2D, 3D,street-view, and/or satellite maps provided mapping services. Thisallows for identification of buildings and the like based on their knownlocations.

In an embodiment, the classification of suspicious objects is based inpart or in whole on a predetermined list of suspicious objects. Forexample, objects predetermined to be suspicious may be, but are notlimited to, other vehicles on the road, pedestrians, animals, debris onthe road, and so on. The list can be dynamically updated. Typically,objects that are in close-proximity and may pose a danger or hazard areclassified as suspicious. It should be noted that a suspicious objectcan be classified as “suspicious and stationary” or “suspicious andnon-stationary”. In a further embodiment, an object may be considered tobe in close-proximity when the object is within a predetermined distance(e.g., 50 meters) from a particular point or other object. As anexample, for an apparatus affixed to a vehicle, an object may be inclose-proximity when the object comes within 75 meters of the vehicle.

At S845, it is checked, for each object classified as stationary, if thedistance to the object classified as stationary is known. If so,execution continues with S847; otherwise, execution continues with S850.At S847, the distance to the stationary object is retrieved fromprevious calculations. This distance information can be derived frompreviously generated 3D maps. A stationary object may be determined tobe stationary relative to the previously generated 3D map. As anexample, if a passive sensor is not moving, an object is stationary, anda previously determined passive measurement and/or active measurementdistance is available for the object from that passive sensor, then thecurrent passive measurement for the object may be determined to be thepreviously determined distance.

At S850, passive measurements are performed based on the acquiredimages. As noted above, such measurements may be performed if theinformation cannot be derived from previously generated 3D maps. Thepassive measurements may include, but are not limited to, measuring atleast a distance from any or all of the classified objects. In anembodiment, the distance from each object classified as “stationary” or“non-suspicious and non-stationary” is determined via the passivemeasurements. The passive measurement may include determining thedistance to an object in the acquired image using a known size (basedon, e.g., number of pixels) of the object, a distance from a validreference point, a vanishing point and vanishing line, a movement of anapparatus or vehicle, and so on. The distance measurements may be savedin a memory (e.g., the memory 215). It should be noted that eachdistance measurement may be for all or any of the pixels composing theobject.

In another embodiment, the passive measurements may be performed basedon movements of objects classified as non-stationary and non-suspicious.To this end, passively measuring the distance may include, but is notlimited to, determining a boundary speed of the object and a directionof movement of the object. The boundary speed is a speed at which theboundary (i.e., outer surface) of the object is traveling. Based on thedetermined boundary speed and direction, a distance to the object may beestimated. In an embodiment, the estimated distance may be further basedon a previously known distance to the object and/or previously generated3D-maps including the object.

At S860, active measurements are performed. In an embodiment, the activemeasurements may be performed by using an active sensor (e.g., theactive sensor 220). Performing the active measurements may include, butis not limited to, measuring a distance from any or all of theclassified objects. In an embodiment, the active measurements may beperformed for objects classified as “suspicious”, “non-stationary”,and/or “suspicious and non-stationary”. In another embodiment, thedistance is actively measured when a reliable passive measurement ofthat distance cannot be achieved. Examples where reliable passivemeasurements cannot be achieved include when, e.g., one or more objectsand/or details in the acquired image appear several times; the imagecontains plain areas with no definite texture (e.g., walls and sky);details in the image are hidden in another view; and the image containsone or more noisy areas. By re-measuring such objects, the accuracy isincreased, and the number of false alarms is significantly decreased.

In an embodiment, S860 includes emitting a laser beam (or a light pulse)using a laser diode. As noted above, the timing, direction and energylevel for each emitted laser beam is controlled to achieve an accuratedistance measurement. In an embodiment, at least the direction of thelaser beam is adjusted based on the time and location difference betweenthe time that an image was acquired by the passive sensor and when theactive measurement has been triggered. This is performed in order tocompensate for the movement of the active sensor and/or a target objectduring that time. The direction in which to point the active sensor canbe estimated using accelerometer information by determining a currentlocation of the active sensor relative to its location when the imagewas acquired.

The emitted laser beam hits a target and a portion of the laser's energyis reflected back toward the active sensor. The returning signal isdetected and the time elapsed between emission of the light pulse fromthe laser and the detection of the returned signal is determined. Adistance measurement of the distance to the object may be determinedbased on the elapsed time. It should be noted that the distancemeasurement may be for any or all of the pixels in images. In anembodiment, multiple active measurements may be performed per object,where at each such measurement, a laser pulse is directed at a differentangle. A resolution of the measurement may be based on the number ofmeasurements performed for each object.

It should be noted that the active measurements performed at S860 may bebased on other active transmissions or emissions such as, but notlimited to, radar, sound, laser triangulation, and the like. It shouldalso be noted that S850 and S860 can be performed in parallel. Forexample, active measurements can be performed in parallel with passivemeasurements on objects classified as non-stationary and/or objects thatcannot be passively measured.

In certain embodiments, the energy level of the emitted laser for eachobject is controlled based on the proximity of the object to the activesensor. In a further embodiment, the closer in proximity the object isto the active sensor, the lower the energy level required. The proximitycan be determined using previous active and/or passive measurements inthe area being scanned. By controlling the energy level, the energyconsumption of the active sensor may be reduced as compared to alwaysusing laser beams having the highest possible amount of energy. Further,controlling the energy levels reduces the risk of hurting humans via thelaser beams as objects that are in close proximity will be scanned withrelatively low energy levels.

At S870, a 3D map is generated based on the passive and activemeasurements. In order to generate a 3D-map of the scene, a distancevalue for pixel in the acquired image should be available. In anembodiment, one or more distance values may be included in the 3D mapfor each segment in the segmentation map. Each distance value can be,e.g., one of the passive or active measurements. In a preferredembodiment, at least 3 distance values for at least 3 pixels of eachsegment should be included in the 3D map.

As noted above, a segment may be part of an object or may be the entireobject. The distance values of all pixels in a segment can be computedor extrapolated based on the at least 3 distance measurements for thesegment. To this end, in an embodiment, the plane equation is computedbased on at least 3 distance measurements for at least 3 pixels in eachsegment. The computed plane equation may be utilized to determine allpixels in the same segment and/or on the same surface. It should benoted that other geometric techniques can be utilized to the compute thedistance values of all pixels in a segment without departing from thescope of the disclosure. It should be further noted that, for somesegments, distance values for all the pixels will be available throughthe active or passive measurements and, therefore, there is no need tosolve the plane equation for such segments.

In another embodiment, if distance values of all of the pixels cannot beestimated based on the plane equation, more active measurements will betriggered.

In yet another embodiment, if all distance values of all pixels areavailable from the previous frame, a limited number of activemeasurements (e.g., 3 active measurements) may be performed and thedistance values from the previous frames may be adjusted based on theactive measurements.

The rendering of the 3D-surface of each segment causes rendering of 3Drepresentations of the identified objects, thereby resulting in a 3D-mapof the scene. The resulting 3D-map can be later utilized to, e.g.,provide driving directions to a controller controlling the operation ofthe vehicle.

It should be noted that the embodiments disclosed herein are describedwith respect to particular units of measurement merely for simplicitypurposes and without limitations on the disclosed embodiments. Otherunits of measurements may be utilized without departing from the scopeof the disclosure.

The various embodiments disclosed herein can be implemented as hardware,firmware, software, or any combination thereof. Moreover, the softwareis preferably implemented as an application program tangibly embodied ona program storage unit or computer readable medium consisting of parts,or of certain devices and/or a combination of devices. The applicationprogram may be uploaded to, and executed by, a machine comprising anysuitable architecture. Preferably, the machine is implemented on acomputer platform having hardware such as one or more central processingunits (“CPUs”), a memory, and input/output interfaces. The computerplatform may also include an operating system and microinstruction code.The various processes and functions described herein may be either partof the microinstruction code or part of the application program, or anycombination thereof, which may be executed by a CPU, whether or not sucha computer or processor is explicitly shown. In addition, various otherperipheral units may be connected to the computer platform such as anadditional data storage unit and a printing unit. Furthermore, anon-transitory computer readable medium is any computer readable mediumexcept for a transitory propagating signal.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the principlesof the disclosed embodiment and the concepts contributed by the inventorto furthering the art, and are to be construed as being withoutlimitation to such specifically recited examples and conditions.Moreover, all statements herein reciting principles, aspects, andembodiments of the disclosed embodiments, as well as specific examplesthereof, are intended to encompass both structural and functionalequivalents thereof. Additionally, it is intended that such equivalentsinclude both currently known equivalents as well as equivalentsdeveloped in the future, i.e., any elements developed that perform thesame function, regardless of structure.

1-27. (canceled)
 28. A machine-readable storage medium encoded with anon-transitory program code for execution by a data processor forgenerating a 3D map of a scene, the program code configured for: a.receiving active sensor data generated by an active sensor performingactive measurements of a scene including one or more target objects,including: i. a transmitter configured to emit energy pulses toward thescene, ii. a receiver configured to detect energy pulses reflected fromthe scene to generate in response to the reflected pulses an activesensor representation of the scene including active sensor dataconveying a plurality of distance measurements, b. receiving image datafrom a 2D image sensor, the image data including a plurality of pixelsconveying a 2D scene representation, c. processing the image data toperform image recognition on the 2D scene representation to identify theone or more target objects in the 2D scene representation, d. fusing theactive sensor representation and the 2D scene representation forproviding a 3D map of the scene including a succession of data frames,each data frame associating distance measurements from the active sensordata with pixels from the image data.
 29. A machine-readable storagemedium as defined in claim 28, wherein the program code is furtherconfigured for processing the 3D map of the scene to derive directionsto operate a vehicle and output the directions to a controllercontrolling an operation of the vehicle.
 30. A machine-readable storagemedium as defined in claim 28, wherein the program code is configuredfor rendering the 3D map.
 31. A machine-readable storage medium asdefined in claim 28, wherein the active sensor is selected from thegroup consisting of LIDAR, RADAR and acoustic sensor.
 32. Amachine-readable storage medium as defined in claim 28, wherein theactive sensor data is mapped to the image data wherein the fusing of theactive sensor representation and the 2D scene representation includesassociating distance measurements of the active sensor data tocorresponding pixels of the image data according to the mapping.
 33. Amachine-readable storage medium as defined in claim 28, wherein thefusing including estimating for one or more pixels, for which nodistance values are available in the active sensor data, distance valuesand associating the estimated distance values to the one or more pixelsin the 3D map.
 34. A machine-readable storage medium as defined in claim33, wherein the fusing including, for first pixels in the image data forwhich no distance measurements are available in the active sensor data,estimating distance values at least in part on a basis of distancevalues in the active measurement data available for second pixels in theimage data that share a common characteristic with the first pixels. 35.A machine-readable storage medium as defined in claim 34, wherein thecommon characteristic includes pixel color, wherein the first pixelshave substantially similar colors than the second pixels.
 36. Amachine-readable storage medium as defined in claim 28, wherein theprogram code is further configured for controlling an energy level ofpulses emitted by the transmitter, wherein the energy level is one of atleast two available non-zero energy levels and wherein the energy levelof pulses emitted by the transmitter is based at least in part on aproximity of one or more of the target objects to the active sensor. 37.A machine-readable storage medium as defined in claim 36, wherein thecontrolling of the energy level including decreasing the energy levelwith an increasing proximity of the target object to the vehicle.
 38. Amachine-readable storage medium as defined in claim 36, wherein thecontrolling of the energy level including increasing the energy levelwith a decreasing proximity of the target object to the vehicle.
 39. Amachine-readable storage medium as defined in claim 28, wherein theprogram code is configured for performing classification on the one ormore target objects identified in the 2D scene representation.
 40. Amachine-readable storage medium as defined in claim 39, wherein theclassification includes classifying the one or more target objects inpre-determined categories of objects.
 41. A machine-readable storagemedium as defined in claim 36, wherein the program code is configuredfor deriving the proximity of the one or more target objects to theactive sensor from the 3D map.
 42. A machine-readable storage medium asdefined in claim 34, wherein the program code is configured to definesegments in the image data based at least in part on the commoncharacteristics, whereby pixels in a given segment share the commoncharacteristic.
 43. A machine-readable storage medium as defined inclaim 42, wherein the program code is configured to identify three ormore distance measurements in the active sensor data associated withpixels in a particular segment and estimates on the basis of theplurality of distance values by using a plane equation, distance valuesfor other pixels in the particular segment for which no distance valuesare available in the active sensor data.
 44. A machine-readable storagemedium encoded with a non-transitory program code for execution by adata processor for generating a 3D map of a scene, the program codeconfigured for: a. receiving active sensor data from an active sensorfor performing active measurements of a scene including one or moretarget objects, the active sensor including: i. a transmitter configuredto emit energy pulses toward the scene, ii. a receiver configured todetect energy pulses reflected from the scene to generate in response tothe reflected pulses the active sensor data including a plurality ofdistance measurements and conveying an active sensor representation ofthe scene, b. receiving image data generated by a 2D image sensor,including a plurality of pixels, and conveying a 2D scenerepresentation, c. perform image recognition on the 2D scenerepresentation to identify the one or more target objects in the 2Dscene representation, d. fuse the active sensor representation and the2D scene representation of the scene for providing a 3D map of the sceneincluding a succession of data frames, each data frame associatingdistance measurements from the active sensor data with pixels from theimage data, e. derive from the 3D map of the scene a proximity of one ormore of the target objects to the active sensor, f. control at least inpart on a basis of the proximity a parameter of the energy pulsesemitted by the transmitter.
 45. A machine-readable storage medium asdefined in claim 44, wherein the parameter of the energy pulses emittedby the transmitter includes energy level, direction, and timing.
 46. Amachine-readable storage medium as defined in claim 44, wherein theprogram code is configured for processing the 3D map of the scene toderive vehicle operation directions and output the directions to acontroller controlling an operation of the vehicle.
 47. Amachine-readable storage medium as defined in claim 44, wherein theprogram code is configured for rendering the 3D map.
 48. Amachine-readable storage medium as defined in claim 43, wherein theactive sensor is selected from the group consisting of LIDAR, RADAR andacoustic sensor.
 49. A machine-readable storage medium as defined inclaim 43, wherein the active sensor data is mapped to the image datawherein the fusing of the active sensor representation and the 2D scenerepresentation includes associating distance measurements of the activesensor data to corresponding pixels of the image data according to themapping.
 50. A machine-readable storage medium as defined in claim 43,wherein the fusing including estimating for one or more pixels, forwhich no distance values are available in the active sensor data,distance values and associating the distance values to the one or morepixels in the 3D map.
 51. A machine-readable storage medium as definedin claim 43, wherein the program code is configured for, for firstpixels in the image data for which no distance measurements areavailable in the active sensor data, estimating distance values at leastin part on a basis of distance values in the active measurement dataavailable for second pixels in the image data that are visually similarto the first pixels.
 52. A machine-readable storage medium as defined inclaim 43, wherein the computer system is configured for controlling anenergy level of pulses emitted by the transmitter, wherein the energylevel is one of at least two available non-zero energy levels.
 53. Amachine-readable storage medium as defined in claim 52, wherein thecontrolling of the energy level including decreasing the energy levelwith an increasing proximity of the one or more target objects to thevehicle.
 54. An automated method, comprising: a. receiving with one ormore processors, first sensor data from a first sensor, the first sensordata representing an environment, the first sensor including a 2D imagesensor and the first sensor data including image data providing a 2Drepresentation of the environment, b. performing with the one or moreprocessors image recognition on the 2D representation of the environmentto identify one or more target objects in the 2D representation of theenvironment, c. receiving with the one or more processors, a secondsignal from a second sensor, the second signal including second sensordata representing the environment, d. performing data fusion between thefirst sensor data and the second sensor data to produce a succession ofdata frames, wherein each data frame including data derived from thefirst sensor data and associated to data derived from the second sensordata to provide a 3D representation of the environment including the oneor more target objects, e. process with the one or more processors thesuccession of data frames to derive driving directions and output thedriving directions to a controller controlling an operation of a vehicledriving in the environment.
 55. An automated method as defined in claim54, wherein the second sensor data conveys distance information,including distance information to the one or more target objects.
 56. Anautomated method as defined in claim 55, wherein the second sensor isselected from the group consisting of LIDAR sensor, RADAR sensor andacoustic sensor.
 57. An automated method as defined in claim 56, whereinthe image data includes a plurality of pixels, the data fusion includingassociating in the data frame pixels with distance measurements derivedfrom the second sensor.